Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Infrastructure: sceNetInet and sceNetResolver support. #18578

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

JustAWhiteBloodCell
Copy link

@JustAWhiteBloodCell JustAWhiteBloodCell commented Dec 18, 2023

Description

Support infrastructure mode in some(?) games by implementing a portion of the sceNetInet and sceNetResolver modules. Haven't tested for longer than 5 mins in game but I don't think there are memory issues etc, it is a fairly straightforward shim.

Summary of changes made

  • sceNet: Move sceNetInet and sceNetResolver into dedicated files for maintenance.
  • sceNetInet: Support most inet functions, excluding those which are not executed for the SOCOMs. Will look into more games.
  • sceNetResolver: Took anr2me's existing PR and incorporated it into the larger refactor of sceNet. There is also some work to add DNS aliasing a la CNAME in order to support revived infrastructure servers. Hopefully these revived services will decide to open source their code to make sure people in the future can set up their own when they inevitably shut down.

Changes yet to make

  • Move away from SceNet prefixing for SceSocket and similar. Thinking just InetSocket and similar.
  • Lots of TODOs in the code still.
  • Core/Net/* should be moved to Common/Net.
  • Use spaces instead of tabs. (my IDE defaulted to tabs I guess? it was a new installation)
  • Ensure braces are unified in all new code with preferred style.

Caveats, assumptions, callouts

  • Socket file descriptors in game are not the same as the native file descriptors. I'm somewhat concerned about using the same descriptors as the host. I would like to switch to using the native UID and actually at least emulate those but there's no guarantee that's possible or practical.
  • Primarily developed (and consequently tested) on Linux platform.
  • FreeBSD, OpenBSD and macOS should fall under the same path as Linux since they treat non-blocking sockets the same (as far as I can tell). Will test on at least FreeBSD or macOS before finally submitting.
  • During WIP: Windows does not work well, the behavior seems off and it seems to be making way too many calls and behaving overall strangely. I think there is a timeout that needs to be adjusted somewhere or something, will do before marking this as not-a-draft.

if (error != ERROR_WHEN_NONBLOCKING_CALL_OCCURS && error != 0) {
INFO_LOG(SCENET, "Requested sceNetInetGetErrno %i=%s", error, strerror(error));
}
switch (error) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Create a map for O(1) lookups.


int sceNetInetTerm() {
ERROR_LOG(SCENET, "UNTESTED sceNetInetTerm()");
SceNetInet::Shutdown();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Figure out why CLion felt like using tabs instead of spaces ......

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was just gonna comment on that :D

@hrydgard
Copy link
Owner

This is, of course, extremely cool! Nice job!

static constexpr int gFdsBitsCount = 8 * static_cast<int>(sizeof(fdMask));

struct FdSet {
fdMask mFdsBits[256 / gFdsBitsCount];

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: document 256 being max FD_SET size for PSP

#pragma comment(lib, "ws2_32.lib")
#define close closesocket
#define ERROR_WHEN_NONBLOCKING_CALL_OCCURS WSAEWOULDBLOCK
using netBufferType = char;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: document why this is necessary, C++ reinterpret_cast dealing with differing C headers between winsock and sane POSIX sockets implementations

@LunaMoo
Copy link
Collaborator

LunaMoo commented Dec 19, 2023

Use spaces instead of tabs. (my IDE defaulted to tabs I guess? it was a new installation)

Don't you mean the other way around? It uses spaces now, while the rest of the project and probably most of us in general uses tabs.

@JustAWhiteBloodCell
Copy link
Author

I actually did mean what I said, but I realized after I posted that tabs was in fact used for this project. I am just too used to spaces instead of tabs 😅. I'll make sure that that is fixed.

@anr2me
Copy link
Collaborator

anr2me commented Dec 19, 2023

Socket file descriptors in game are not the same as the native file descriptors. I'm somewhat concerned about using the same descriptors as the host. I would like to switch to using the native UID and actually at least emulate those but there's no guarantee that's possible or practical.

PSP only supports socket id/file descriptors of 0 to 255 (8 bits), some games even use the lowest 8-bit value only, if you use Windows's socket fd directly without mapping it to [0..255] many games will be facing crashes (due to out of bound access during FD_SET/etc. by the game) and inaccuracy during select (since fd number higher than 255 can't be mapped into the 256-bits bitmap), as PSP use a bitmap of socket's fds (similar to Linux/BSD) instead of array of fds like what Windows's use. Linux should also face the same issue as it supports more than 255 (1023 or more i think) fd number.
And as i remembered these 0..255 range ids also shared with Adhoc's sockets (ie. creating PTP/PDP sockets will increase the id number too)

You should create a socket objects and map the id there, you will also need to store non-blocking information and implement a simulation of blocking behavior instead of using blocking mode directly (which will blocks the emulation).
Simply replacing blocking socket with non-blocking one without handling/converting the error codes properly may not always works on some games (similar issue happened before blocking simulation being implemented on Adhoc sockets in the past)

@JustAWhiteBloodCell
Copy link
Author

JustAWhiteBloodCell commented Dec 19, 2023

Thanks for that, I think I saw similar comments by you (and others) elsewhere and so that's what I've done.

  1. socket ids are base-1, controlled by mCurrentSocketId in SceNetInet class which is to say that when inet is re-initiailized the number will reset and there is a mapping stored to provide quick translation.
  2. I have created SceSocket which contains non-blocking information and the native socket id. I will most likely rename this class to InetSocket.
  3. Non-blocking functionality is implemented as you describe.
  4. I've also translated the PSP 256-socket-limited fd_set into the platform-specific fd_set specifically for select and I've observed it to work. I also have a check primarily meant for Windows to ensure that the max translated fd_set is not too large. I find it very unlikely that a PSP game would have more than maybe a dozen sockets open at once though.
  5. Currently working on the error code translation. I did at least translate the get/setsockopt optnames so far.

If you have any examples of such games please let me know because I'd like to validate its robustness as much as possible.

@anr2me
Copy link
Collaborator

anr2me commented Dec 19, 2023

I'll upload my stashed code later after i fixes the conflicts with the latest master, it probably not the latest one i had, since i couldn't found the latest one where i already fix the conflicts and separating the NetResolver out. (Edit: Looks like NetResolver has been separated out from this stashed code, but still have the sceHttp included)
I remembered putting my whole Projects folder on a single RAR file before i lost all my partitions due to bad sectors, but couldn't found that RAR file on my external drive :( may be i forgot to copy that file to external drive.

Anyway, you can use my stashed code as reference if you want, as i already implement nearly all the Inet syscalls (i think i only missing the SceNetInetSendmsg, but you can use my SceNetInetRecvmsg implementation as reference, both of them have a complex structure containing flags that need to be converted, but only 2 games so far known to use such syscalls)

Edit: ugh i'm facing an error i don't know how to resolve after i'm done fixing the conflicts.

Severity	Code	Description	Project	File	Line	Suppression State	Details
Error	RC2104	undefined keyword or key name: PPSSPP_WIN_VERSION_STRING	PPSSPPWindows	D:\Data\Projects\ppsspp\Windows\version.rc	28		

PS: i'm using a fresh installation of Win11Pro 23H2 + Visual Studio Community 2022 (Desktop C++ Dev & UWP components) + Python 3.12.1, did i missed anything else?

Edit2: Oops i forgot that i use the embedded version of git in Sourcetree, thus not in the PATH env var (didn't know that the build process will calls git command)

@JustAWhiteBloodCell
Copy link
Author

JustAWhiteBloodCell commented Dec 20, 2023

Thanks for that! You don't need to resolve conflicts etc I can handle cherry-picking it in even if its just a patch file and include you as a co-author, if that works for you? I plan to resume working on this weekend which is when my vacation starts.

@anr2me
Copy link
Collaborator

anr2me commented Dec 20, 2023

I've uploaded it as PR at my fork anr2me#21

The reason i need to fix the conflicts (and also merge back the NetResolver library) was because i need to build an up to date test builds for testers at #14256

I also need to create new yml file to generate IPA for iOS later (taking it out from #15450)

@JustAWhiteBloodCell
Copy link
Author

Thank you! I should be able to get this PR updated against it this weekend and then move onto fixing all the builds..

@hrydgard
Copy link
Owner

hrydgard commented Jul 19, 2024

Is this in a state where we can just get it in as-is, without breaking other stuff? Got a report that it works fine for Wipeout Pulse and that community are going to use it through their own builds regardless, heh.

(Yeah, the description says it's a bit wonky on Windows, but it might be okay)

@ChaCheeChoo
Copy link

Is this in a state where we can just get it in as-is, without breaking other stuff? Got a report that it works fine for Wipeout Pulse and that community are going to use it through their own builds regardless, heh.

(Yeah, the description says it's a bit wonky on Windows, but it might be okay)

This is not the build we use for Pulse. We use https://github.com/anr2me/ppsspp/tree/infra2 instead. Specifically the infra2 branch.

@anr2me
Copy link
Collaborator

anr2me commented Jul 20, 2024

Is this in a state where we can just get it in as-is, without breaking other stuff? Got a report that it works fine for Wipeout Pulse and that community are going to use it through their own builds regardless, heh.
(Yeah, the description says it's a bit wonky on Windows, but it might be okay)

This is not the build we use for Pulse. We use https://github.com/anr2me/ppsspp/tree/infra2 instead. Specifically the infra2 branch.

That mean it's PR anr2me#21

Even though my code have more syscalls implementation that this PR, but it's not well written (ugly) compared to this.
And this PR already stores the socket's stat on the faked PSP socket struct, while mine still using the host's socket directly, thus have an incompatible socket id issue and can't differentiate whether it was a UDP or TCP socket later when the socket used (which is why i'm opening both UDP & TCP on UPnP, which is kinda bad)

@hrydgard
Copy link
Owner

@anr2me Then maybe it makes sense to merge this soon, and then for you to rework your PR on top? Or what do you think?

@anr2me
Copy link
Collaborator

anr2me commented Oct 22, 2024

I'm afraid i'm no longer working on network stuff, not even adhoc stuff.
I can probably help fixing bug that need a small changes.
Hopefully, @JustAWhiteBloodCell or someone else can continue it based on this PR.

@hrydgard
Copy link
Owner

hrydgard commented Oct 22, 2024

Ah yeah, that's totally understandable of course.

In that case, the best next step might be to just get this in (after the 1.18 release)

@hrydgard hrydgard added this to the v1.19.0 milestone Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants